Face Finder: Filtering a Large Face Database using Scars, Marks and Tattoos

Facial marks (e.g. moles, freckles, and scars) are soft biometric traits that have limited utility in uniquely identifying individuals. However, facial marks can still play a crucial role both in filtering a large face gallery, as well as in distinguishing between ambiguous face comparisons such as identical twins. This research demonstrates that facial marks can indeed be used to speed up face matching and assist in individualizing identical twins. The facial mark detection process consists of two main stages: (i) Landmark detection using Active Shape Model (ASM) to represent face images in a common coordinate system, and (ii) a scale space extrema detection method to detect facial marks. All the detected marks are classified into six different types based on their morphology and color, and are encoded into a fixed length feature vector to facilitate efficient matching and retrieval. Experimental results on face retrieval using a mug shot data set consisting of 1,000 probe images (one per subject) and 100,000 gallery images (one per subject) demonstrate a significant improvement in retrieval time with only a slight loss of matching accuracy. We also demonstrate a symbolic query based retrieval capability using the facial mark based indexing scheme. The symbolic queries can be constructed based on the morphology, color, and location of a mark. This will facilitate the face retrieval process when only a verbal description of the face is available instead of a the face image as a query. Additional experiments on a twin data set of 404 images collected from 178 identical twins (89 twin pairs) further demonstrates improvements in differentiating identical twins using face marks.